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PREFACE 


The work described in this report was performed by the Information 
Systems Division of the Jet Propulsion Laboratory. 
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ABSTRACT 


This paper introduces a globally adaptive image compression structure 
for use in a tactical RPV environment. The structure described would 
provide an operator with the flexibility to dynamically maximize the usefulness 
of a limited and changing data rate. The concepts would potentially simplify 
system design while at the same time improving overall system performance. 
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RPV APPLICATION OF A GLOBALLY ADAPTIVE 
RATEXONTROLLED COMPRESSOR 


I. INTRODUCTION 

The purpose of this report is to briefly introduce a globally adaptive 
rate controlled image compression structure which is ideally suited to the 
tactical environment of a Remotely Piloted Vehicle (RPV). Such a structure 
could provide an operator with the flexibility to dynamically maximize the 
usefulness of a limited and changing bit rate. This structure was previously 
developed as a fundamental part of image compression algorithm RM2^J^' 
Required modifications to standard algorithms are noted. 


RPV Environment. Simplified 

The assumed tactical problem is one in which an operator-controlled 
RPV equipped with an imaging device flies over terrain looking for various 
targets. An operator views imaging information transmitted back and first 
seeks to detect and recognize targets of interest. Once accomplished, the 
operator may initiate automated tracking and/or various actions against 
the targets. 

A sophisticated adversary will seek to jam both the command uplink and 
imaging downlink of the RPV communication system. We will assume that 
a necessarily digital spread spectrum anti-jam (AJ) downlink is used to 
counter such jamming threats. For our purposes here we will also assume 
that the AJ protection is sufficient to provide an error-free uplink. A 
simplified picture of the operational environment that results is shown in 
Fig. 1. 

^Performance curves for an adaptive cosine algorithm were mislabeled in 
Ref. 2 as adaptive Fourier. 
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UPLINK MOD' COMMANDS 



RECEIVER 


Fig. 1. Simplified Block Diagram. 

As shown in Fig. 1 the error rate at the ground station receiver is 
automatically monitored. When the error rate exceeds some threshold, an 
uplink command is given to reduce the available data rate (to gain more AJ 
protection) until the received error rate is acceptable (this could be an 
automated or operator controlled function). Similarly if the jamming 
situation improves the available data rate is increased. 

Thus the operator is faced with accomplishing his task of detection, 
recognition, etc. , under limited and changing data rate conditions. He must 
dynamically communicate commands to the imaging data system to select 
modes which maximize the usefulness of the limited data rate. 
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II. RM2 SYSTEM STRUCTURE 

This section provides a tutorial view of the RM2 algorithm internal 
structure and a description of the flexibility it provides in utilizing a 
limited number of bits more effectively in a given image frame. A later 
section will be concerned with extending this flexibility to sequences of 
images within an RPV tactical environment. 

Bits/Pixel vs. Quality 

In simple terms RM2 can compress images to any requested bits per 
picture element (bits/pixel). A higher requested bits/pixel results in a 
better decompressed approximation to the original image. This basic 
relationship is shown in Fig. 2. 



0 1.0 2.0 3.0 4.0 

BITS/PIXEL i 


Fig. 2. Monotonic Relationship Between 
Bits/Pixel and Image Quality 
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Internal Bit Allocations 


RM2 treats an image as an array of subpictures (e. g. , 32 x 32 
pixels) and first surveys these subpictures to determine thei: "data 
activity" A^, A^, . . . A^ as shown in Fig. 3. These measures of activity 
directly relate to the relative need of each subpicture 
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Fig. 3. Array of Data Activities. 

for the limited number of bits available for the complete image. An 
internal algorithm ^ ^ will allocate the total number of bits available 
for an image frame, B, to the subpictures in a way that reflects this need. 
The more active subpictures receive more bits that the less active sub- 
pictures. As in Fig. 2 more bits applied to a given subpicture imply 

th 

better quality. Let b^, i = 1, 2, . . . , N be the bits assigned to the i — 
subpicture, then 

B=^Pb. (1) 
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Rate distribution parameter . The rate allocation algorithm described 
in Ref. 1 includes a parameter which controls the amount of nonuniformity 
of rate distribution. At its lowest setting, this parameter causes the same 
bits/pixel to be allocated everywhere, regardless of any differences in 
the activities. As this parameter is increased the range of bits/pixel 
allocations also increases (provided the activities aren't all the same). 

The consequence of this effect is that very low parameter settings put a 
"quality emphasis" on inactive subpictures relative to active ones whereas 
a very high setting emphasizes active subpictures. 


Edit header. Define an edit header 


E = e l e 2 ' • • e N 


( 2 ) 


where is a one bit indicator of the editing status for the i— subpicture in 

the image. If e^ = 1 then the i— subpicture is treated normally, whereas if 
th 

e^ = 0 then the i — subpicture will be "edited" and the allocation b^ is only one 
bit (the header bit). If e^ is zero, then the ground decompressor would repro- 
duce subpicture i as all zero. The bits that would have been used on edited 
subpictures are instead allocated (still according to relative activity), to the 
non-eUited subpictures. This means that different frame sizes can be speci- 
fied by sel cting a different header, as for example in Fig. 4. 


Enchanced Q for Selected Areas 

Suppose it is desirable to emphasize the quality in a selected area 
while still limited to a fixed bits/image. This can bo accomplished quite 


5 




EDITED SUBPICTURES 



Fig, 4. Changing Frame Size by 
Editing Subpictures. 


simply by artificially increasing the activity measurements for the selected 
subpictures. The algorithm that allocates bits would cause more bits (and 
hence quality) to flow to these subpictures at the expense of the subpictures 
*:hat did not get an artificial increase to their activity. 


Foveal spot . The most obvious application of selective quality enhance- 
ment is the concept of a foveal spot where an operator selects the central 
part of an image for emphasis as in Fig. 5. 

ARTIFICIALLY 
INCREASE ACTIVITY 



Fig. 5, Foveal Spot, 
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The size of the foveal spot is of no consequence, other than its 
boundaries should correspond to the subpicture boundaries. The tradeoff 
between quality inside or outside the foveal spot is easily varied by pro- 
viding a single parameter, say j3, which simply multiplies the normal 
activity measures before bit allocations are made. 

Cursor foveal spot. It might be desirable to allow the operator to move 
the foveal spot to areas vhich are of specific interest. That is, the operator 
may want excellent quality in some specific region. This might mean moving 
the central foveal spot to an operator controlled cursor or introducing an 
altogether separate region of emphasis. In either case, the implementation 
is no different; multiply the activities of the selected region by some param- 
eter (3'. 

Target screener . While not immediately available in hardware form, 
pattern recognition techniques would provide in the not too distant future 
the ability to automatically select areas which, "with high probability, " con- 
tain targets of special interest. This is much like the foveal spot except the 
spots are automatically selected instead of selected by the operator. 

However, the same concept holds. Given a limited number of bits 
available in a frame time it is clearly desirable to place them where they 
dr the most good. This leads to the most general situation depicted in 
Fig. 6. 

In the figure we show the following: 

a) A selected edited region which reduces frame size. 

b) A selected foveal spot region. 

c) Subimage regions which an automated pattern recognition algorithm 
has determined as likely to contain targets of specific interest. Two 
possible classes are shown. 
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FOVEAL SPOT BORDER 


DETECTED 
CLASS 1 



Fig. 6. General Flexibility. 

In actual situations, a data rate limitation forces a tradeoff between 
frame rate, frame size and the bits available per frame time. As discussed 
earlier: 

1) the specified edited regions would receive no bits (except for the 
header bits); 

2) activity measurements would be made in non-edited areas; 

3) activities inside the foveal spot would be increased by some factor, p 

4) areas identified as target class 1 would get an added boost say y^, 
and class 2 area activities would be boosted by The values of 
Yj, y ? no doubt reflect the importance of target classes 1 and 2 
but would be experimentally determined;^ 

5) bits are allocated to subimages according to the relative activity 
measures; 

6) actual compression coding of the image is performed. 

^Observe that the target screening need not be applied only to non-edited 
regions. If a "target” was determined to be likely in an edited area „he 
edit command could easily be overridden in that specific area. 
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In a real system, various edit headers, foveal spot definitions and 
activity emphasis parameters could be stored in ROMS and selected by 
appropriate system commands. 

The combination of a target screener, once practical, with a compres- 
sion algorithm which is structured like RM2 should result in significant reduc- 
tions in the required data rate needed to accomplish various mission objec- 
tives. A flow chart of these simple steps is shown in Fig. 7. 

Modifications to Standard Algorithms 

If we delete the ability to distrii. ite a given number of bits according to 
a "data activity" measure then the more familiar standard two-dimensional 
algorithms can be simply modified to accomplish most of the features 
described in previous paragraphs. The required attributes are as follows: 

a) The algorithm should be structured to operate on subpictures (this 
is usually inherent anyhow). 

b) Rather than be limited to one or two compression options, the 
algorithm should provide the ability to choose closely spaced bits/ 
pixel options for each subpicture which are monotonically related 
to the quality of reproduction of a given subpicture.* Each coded 
subpicture would be prefixed by a header which identified the 
selected option. The range of options should be broad (e.g. , 

0. 5 bits /pixel to 2. 5 bits /pixel). 

These are not difficult requirements and would permit most of the rate 
distribution capabilities just discussed: editing, arbitrary foveal spot empha- 
sis (cursor or center), and target screener emphasis. The same rate alloca- 
tion algorithms could be used by assuming activities were all unity. 

*See Ref. 1 for simple RM2 control loops to handle cases where the allocated 
number of bits does not equal a prescribed option. Basically, any discrep- 
ancy is reallocated to subsequent pictures or subpictures. 
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Fig. 7. Generalized Bit Allocation Flow Chart. 
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Computation. The conversion of standard algorithms to the desired rate 
control structure would have a minor impact on overall computation require- 
ments. Adequate activity measures can be obtained using only a few adds and 
subtracts per pixel. Further, the per pixel computations needed to determine 
subpicture bit allocations are negligible. 

Adding activity measures . Actually, it is a simple matter to obtain 
activity estimates quite similar to those derived in RM2. An estimate of 
data variance, or entropy would suffice. Any of these, properly normalized, 
would permit a direct use of the RM2 or similar rate allocation and control 
algorithms. All of the preceding discussions would hold. The only difference 
would be the manner in which a given subpicture is represented at a specified 
bits /pixel. 

HI. DATA SYSTEM COMMAND AND CONTROL 

The preceding discussion should indicate that a globally adaptive rate 
controlled structure can provide extensive flexibility to make the most of a 
limited number of bits available for a static image frame. Here we extend 
discussions to the dynamic situation in which an operator is making real-time 
decisions on how he will make most effective use of a limited and changing 
data rate. 

Parameters 

Frame rate, 30 /k. Assume that all possible frame delays, k/30, lying 

between k . / 30 and k / 30 are options. (This indirectly specifies frame 
min max r 7 r 

rates f^ = 30/k. ) 

k k +1 . k 

min mm k ... max 

-30~' "30 ' TO ' 30 < 3) 
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Frame size, FS. A collection of frame sizes labeled FS = FS^ where 
FS^ equals the number of pixels in frame size i (assumed to be a multiple 
of the subpicture size). Assume also that the FS^ are monotonically ordered 
so that 


FS < FS, < FS, < . . . < FS. 

max 

Increasing or decreasing frame size will mean stepping through the FS.. in 
this order. 

Foveal spot, FV. These are similarly defined as FV. where 

3 


FV q < FV X < FV 2 < . . . < FV. 


max 


(5) 


Bits /pixel, b. For later use we specify parameters 


b 


max 


and b 


min 


( 6 ) 


as the maximum and minimum desired average bits /pixel, b, in a frame, 
bfnax correspond to the compression rate at which no noticeable degrada- 

tion is possible and b to a minimum acceptable quality under severe 
conditions. 

Data rate. Let D = be the possible data rates available where 

D 0 < D 1 < °2 <* * * D d (7) 

max 

While an operator could choose D, it is more reasonable to assume that D is 
automatically determined by monitoring the bit error rate at the ground receiver. 
Increasing or decreasing D will mean stepping through the specified in (7). 
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Basic Constraints 


Quite trivially we have the equation 

D = (f3j (FS) (b) (8) 

Since bits/pixel, b can be arbitrarily assigned when using RM2 or some 
other properly structured algorithm, the parameters k and FS are acceptable 
options provided b lies in the allowable range. That is 


b . < b < b 
min max 


(9) 


The only constraint on foveal spot size FV is that it be less than or equal to 
the corresponding choice for frame size FS. 

FV < FS (10) 

We will henceforth assume that any foveal spot command for which (10) is not 
true will automatically be converted to FV = FS (which is equivalent to "no foveal 
spot"). 

Changing System States, Concept 

This section addresses the problem of allowing an operator to efficiently 
move between the potential system states specified by k, FS and b. The concept 
is introduced in Fig. 8. For the moment we will assume that data rate D is 
fixed. 

In the figure an operator has three levers or slide switches at his com- 
mand, one for frame rate, one for frame size and one for quality. The relative 
position of the levers corresponds to an operator's current relative importance 
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Fig. 8. Lever Control of System States. 

that he places on the 3 variables k, FS and b. ^ When the 3 levers are all in 
the same horizontal position then the system will be in some preselected 
reference state defined by 
# * * $ 

S (D) = (k , FS , b ) (11) 

a$c 

where S (D) satisfies (8) at data rate D. This reference state might have been 
determined from a prior run on a target at rate D or simply established as a 
good choice during training. In any case it can be reset at any time to a current 
state by pressing a "state set" button as shown. 

If he moves one of the levers away from the equal emphasis position (all 
switches in the same horizontal position), more (up) or less (down) emphasis 


t Remember k = 30/(frame rate) and b = bits/pixel is monotonically related to 
image quality. 


14 





is placed on the corresponding parameter. For example, as he raises the qual- 
ity lever, the bits /sample would increase at the expense of the other parameters 
frame rate (30/k) and frame size (FS) which would have to decrease to satisfy 
the constraint equation (8). If a limiting value for a parameter is reached 
(i. e. , max or min) no more changes to system states would occur. Such bound- 
ary conditions could be signaled to an operator. An operator can always return 
to the reference state by equalizing the lever positions. 

Fixing single parameter . If an operator wants to fix a particular parame- 
ter and trade off the remaining two he simply presses the appropriate "Parame- 
ter hold button" as shown in Fig. 8, This would fix the selected parameter at 
its current value. Subsequent adjustments to the remaining two lever positions 
would effectively trade off the relative importance of the remaining two parame- 
ters, while still satisfying (8). 

Automated adjustments to b. Instead of fixing the bits /pixel parameter 
b as above, an "activity track" mode could be implemented. In earlier discus- 
sions we described the internal operation of RM2 (or modified standard 
algorithms) as one which uses subpicture activity measurements as a measure 
of the relative need of each area for a limited number of bits, with the more active 
areas requiring more bits than less active ones. Such measurements can be 
used to roughly maintain a given average quality as terrain changes. When an 
"activity track" mode is activated, b would be adjusted up or down in subse- 
quent frames depending on the values of average image activity compared to 
the corresponding value at activation. 

Automated edit. When there is a gradual sweep of ground terrain, much 
of each image may be redundant. Improved quality in the non-redundant image 
areas can be obtained by editing those image areas where frame overlap 
occurs. The internal rate control structure described earlier allows for a 


15 


simple implementation of the editing functions. The information necessary 
for automated decisions of which areas are affected should be available from 



camera gimbal controls, RPV position and motion information. This could 
have a significant impact on capability during panoramic scans. The lever 
controls of frame rate, frame si<-f» and quality (bits /pixel) described in Fig. 8 
would still operate in the same manner. Quality would increase at a given 
frame rate and frame size because bits which would have been allocated to 
edited redundant regions would instead be allocated to "new data" areas only. 
The operator might then elect to increase frame rate or frame size. 

Data rate change s. As noted earlier, data rate D can be expected to 
change as a function of variations in the jamming environment. Such changes 
could be in factors of two. Ideally the system should switch from a set of 
parameters which the operator has optimized for the particular tactical 
situation and rate D, to one which is similarly optimized for the new rate D'. 
In a real system, the closer the new set of parameters is to the ideal, the 
less the operator has to fiddle with knobs, etc. The concept described in 
Fig. 8 is a reasonable approximation to these goals. 

Each data rate D has a corresponding reference state S (D), where 

)je sje 

S (D) has been previously determined as a "good" choice. S (D) might be 
determined as simply a state which is typically good for data rate D. More 
dynamically, S (D) could be selected by the current operator as his best 
choice for the given situation (perhaps as determined by an earlier run at 
rate D). 

If the operator's levers are held in an equal emphasis position (all in 
the same position) then the system would switch between reference states as 
D varied. However, when an operator moves away from S (D) to some new 
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state by moving the levers he is establishing a new relative emphasis between 

the parameters k, FS and b. If D is changed to D* this same relative emphasis 

$ 

is instead applied to move the state away from reference state S (D), That 

$ 

is, the system responds as if the system had been in reference state S (O') 
and the operator had then moved the levers, 

Joy stick. It may be simpler for the operator if the three emphasis 
levers are combined into a single joy stick implementation. 

Parameter Emphasis 

This section provides some additional detail on potential approaches 
to the implementation of state adjustments by parameter emphasis. 

Let 6p, and 6^ be operator assigned priorities for frame rate 
(30 /k), frame size (FS) and b= bits/pixel (quality) respectively, satisfying 

6 r 6 fs 6 b - 1 < 12 > 

Let k , FS and b be parameter values corresponding to reference 
$ 

state S (D). Then new values satisfying (8) can be determined from 

D = <6 f • <6 fs FS*) (6 b b*> (13) 

k 

as 

$ 

k' = FS* = 6 fs FS*, b* = 6 b b* (14) 

F 

Some additional tests are necessary to ensure from 3, 4 and 9 that 

a) k', FS' and b* lie within the allowable boundary values (i. e. , the 
maximum and minimum range), and 

b) The chosen values are actually available options. 
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The boundary conditions on k, FS and b simply mean that the 6 are 


also bounded. For example 


max 


'F < 


rmn 


(15) 


Further motion of a given emphasis lever beyond a boundary value 
would have no effect. That is, the corresponding parameter is fixed at 
the boundary value. Reaching a boundary value can be easily signaled 
to an operator (e. g. , buzzer). 

If the values of k' and FS' as computed in (14) do not match up with the 
allowed values in (3) and (4) they are set to the nearest allowed values, b' is 
adjusted to satisfy (8). It is not necessary that the available options for b 
match up precisely with the computed b'. A simple control loop described 
in Ref. 1 will handle this mismatch provided the available options are in 
steps of 10% or so. 

IV. CHANNEL CONSIDERATIONS 

Error sensitivity problems with transmitted data have generally meant 
reducing transmission rate (increase S/N) to reduce the error rate to an 
acceptable level. A given compression technique will tend to become more 
sensitive as that technique is modified to become more adaptive. Individual 
errors have a more dramatic impact on reconstruction. Errors in identifiers 
of algorithm changer (such as headers) can propagate their effect over long 
sequences. 

A practical solution to this problem was developed for space applications 
in Refs. 1, 3 and 4. Briefly, the solution is to concatenate a (large symbol) 
interleaved Reed-Solomon code with a convolutionally coded, Viterbi decoded 
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inner channel, yielding virtually error free data at the same S/N that an uncoded 
link would have a P e * 1/33. In addition interleaved Reed-Solomon coding by 
itself can provide significant burst error protection. Boyd, Cain and Clarke (5) 
clearly suggest that high rate implementations of the desired RS coders and 
decoders is feasible with today's technology. The RS/Viterbi combination would 
appear to be directly applicable to the various AJ spread spectrum techniques 
likely to be elements of an RPV communication system, 

V. DISCUSSION 

The globally adaptive rate controlled compressor structure clearly opens 
up the practical possibility of providing an operator with extensive flexibility to 
dynamically maximize the usefulness of a limited and changing bit rate. Prelim- 
inary descriptions of possible approaches were presented above. Potentially, 
such techniques should result in significant improvements in overall system per- 
formance within severe tactical environments. 

Standard approaches to system design would require selecting a few param- 
eter options years before they are used. This essentially means trying to pick a 
set of options which would "on-the-average" perform best over all possible 
tactical environments, operators, sensors, data rates, etc. The choices 
would necessarily be only intelligent guesses since testing all the possibilities 
beforehand is a pract. sibility. This perhaps "most difficult" aspect of 

system design is alleviated by the approaches described above since "all" the 
possible options are always available to an individual operator. 

The compressor rate control structure and the operator control approaches 
presented here exhibit some degree of autonomous operation. That is, a limited 
number of bits can be adaptively distributed spatially and temporally to improve 
the return of desired information. Further degrees of autonomous operation 
which further reduce operator intervention should be possible. 
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